The present invention relates in general to systems used to determine information about the type of occupant in a seat. In particular, the present invention uses a rules-based expert system to classify the occupant of a seat into one of several predefined occupant-type classifications so that an airbag deployment system can make the appropriate adjustment with respect to airbag deployment.
Conventional airbag deployment systems have contributed significantly to the safety of occupants in automobile crashes. However, there may be occasions when due to the particular type of occupant in a particular seat, the deployment of an airbag may not be desirable. This may be true even though airbag deployments generally provide substantial safety benefits. Presently, there are manual devices that allow the occupants of an automobile to disable an airbag deployment system. These devices depend on an occupant to remember that the deployment status of the airbag may need to be changed, examine the current status of the airbag deployment system, and decide on the basis of the type of occupant whether or not the deployment system should be enabled or disabled. Ultimately, manual systems are less desirable than automated systems for airbag disablement because the determination of whether an airbag should be disabled can be objectively determined, and is substantially a function of the type of occupant sitting in the seat behind the airbag. It would be desirable for a system to automatically determine whether an airbag should be disabled on the basis of the type of occupant sitting in the seat behind the airbag.
Known automated systems suffer from substantial limitations. One system relies on occupants to carry a data card which needs to be inserted into a scanning system on every trip. Such a system does not provide guest riders with desired protection since a guest rider has no reason to possess such a card. Moreover, the inconvenience to regular riders of a vehicle will still likely impede use of such a system. It would be highly desirable for an occupant classification system to function accurately without any need for affirmative cooperation from the occupant.
Some existing systems rely on weight sensors in the seat to determine the type of occupant in a seat. Weight-based systems suffer from inaccurate weight measurements because the movement of an automobile causes the weight in a seat to shift, and thus makes weight difficult to measure accurately. Moreover, a weight sensor cannot distinguish between a 70 pound child and a 70 pound inanimate object. It would be desirable for an occupant classification system to be based on non-weight based characteristics of the occupant.
It is known that optic sensors can determine the distance between the occupant at certain locations in the seat, and the airbag. Such systems may be superior to weight-based systems, and some hybrid systems may even utilize both weight-based sensor data and optically-based sensor data. Although the attribute of distance between the occupant and the airbag may be a useful attribute to consider in making an occupant-type classification, there are limits to relying solely on position-based data. It would be desirable if even dynamic systems such as those systems monitoring the ever changing position of the occupant, would also incorporate intelligence regarding the visual image of the occupant. It would be highly desirable for an airbag deployment system to know the classification of the occupant based on the image attributes of the occupant.
It is known in the prior art that a visual image taken by a camera, instead of mere data points captured by an optical scanner, can be used to determine whether or not an airbag system should be disabled. However, such systems make an airbag disablement decision by attempting to match the then current occupant image to one of many template images, with each template image associated with a particular airbag deployment decision. In some cases, the template image may even be associated with a predefined occupant classification type that then in turn, determines the particular airbag deployment decision.
Such template-based methodologies suffer from several limitations originating from the number of template images needed to match a voluminous number of potential situations. Occupants sit differently from one another, move differently, drink beverages, eat food, talk on cell phones, and actively pursue all sorts of different behaviors while in automobile seats. The sheer volume of potential variations makes holistic matching a tedious and ultimately inaccurate process because many of the variations are not relevant to the airbag deployment decision, and will only serve as white noise for the template-based system, preventing an inaccurate assessment of the occupant. Resource constraints are also an important limitation, because ultimately all processing will be performed by a small on-board computer. Similarly, performance constraints associated with comparing an occupant image to thousands or more template images are important consideration for a system required to perform its function in fractions of a second. In summary, holistic template-based matching approaches spend too much time and resources being confused by image attributes not likely to be relevant to determining whether or not an airbag should be precluded from deploying. It would be highly desirable if a more focused approach were used, an approach based on the proven predictive value of a particular feature or set of features.
Due to these limitations, it would be desirable for a library of key occupant image attributes (xe2x80x9cfeaturesxe2x80x9d) be used to classify the occupant. It would be advantageous for an occupant classification system to isolate key features instead of being distracted or even misled by the distractions of extraneous xe2x80x9cwhite noisexe2x80x9d images attributes not relevant to a occupant classification decision. Such key features could then be tested for their predictive value with respect to distinguishing between particular occupant types. It would be helpful to construct algorithms utilizing only those features most helpful in an occupant-type classification decision. It would also be desirable for an expert system to implement such algorithms in a timely and automated way. If multiple features were to be utilized to generate a single weighted classification type, it would be desirable for a single weighted confidence factor to be generated for the single weighted classification determination. It would also be desirable for a system to utilize a sufficient number of predefined and comprehensive occupant classification types in order to for the airbag deployment system to make well-informed decisions. Such a system should be able to distinguish between an adult, a rear facing infant seat, a front facing child seat, a booster seat, a child (between 6 and 8 years old), miscellaneous occupants (such as inanimate objects) and indeterminate occupants (none of the previous categories). The use of fewer classification types is undesirable because it impedes the ability to isolate, develop, and key features for use by an expert system classification system.
One final problem common to all occupant classification systems used by airbag deployment systems is the question of accuracy, and how to deal with the inevitable inaccurate determination. Although the goal of such systems is to be as accurate at possible, no system is perfect, and current systems do not provide any means by which the airbag deployment system can weigh the accuracy of a classification made by the occupant classification system. The decision to deploy or disable an airbag should incorporate the probability that an occupant-type classification was accurately made. Thus, it would be helpful if each occupant-type classification were accompanied by a confidence factor indicating the relative accuracy of the occupant-type classification. It would be desirable is such a confidence factor could in part be generated from test results using known images of potential occupants.
The present invention relates to a occupant classification system that utilizes a rules-based expert system to categorize the occupant of a seat into one of several predefined occupant-type categories on the basis of key image attributes (xe2x80x9cfeaturesxe2x80x9d) of the occupant. The occupant classification system then uses a confidence factor extractor to generate a confidence factor associated with the occupant-type classification with a higher confidence factor indicating greater likelihood that a particular classification is correct. Both the occupant-type classification and the associated confidence factor are sent to the airbag controller so that the appropriate steps may be taken with regard to the behavior of the airbag.
Use of the occupant classification system requires that a camera capture the image of the occupant. A feature extractor takes the image captured by the camera, and then extracts the key features of the occupant. A vector of features relating to the visual image of the seat area in a vehicle such as an automobile, plane, train, or boat, and classifies the occupant of that seat into one of several predefined classification categories. Inputs to the inventive system could include the surrounding image of the entire seat area (the xe2x80x9cambient imagexe2x80x9d), the image of the occupant (the xe2x80x9csegmented imagexe2x80x9d) without the image of the environment surrounding the occupant, or simply a vector of features containing information regarding the segmented image or the ambient image. For embodiments where the inputs are the segmented image or the ambient image, the invention contains a feature extractor to extract a vector of features from those images.
Users of the invention can create new features, derive new features from existing features, and test the effectiveness of particular features and particular combinations of features with respect to distinguishing between particular occupant-type classifications. The opportunity to further refine such features and sets of features is a substantial advantage of the invention. This flexibility provides users of the invention with the opportunity to maximize the accuracy of occupant-type classification determinations. Focusing on the key features that are particularly successful at distinguishing between particular occupant classification types facilitates timely and accurate classification determinations, while minimizing resource requirements for the classification system through heightened efficiency.
The present invention also includes a method for determining whether or not a particular feature should be included in a library of features used by an expert system classifier. A set of features can be tested for their aggregate accuracy in correctly predicting occupant-types using an expert system classifier (xe2x80x9cresource classifierxe2x80x9d). Subsequent to testing the aggregate accuracy of the resource classifier, individual features can be removed from the resource classifier, and those results can be compared with the test results from the resource classifier to determine if that feature that was removed was advantageous or disadvantageous to the resource classifier. Each test feature is then put back in the library of features, and a new feature is then removed for testing until each feature in the resource classifier has been similarly tested by that features isolated removal. This methodology is beneficial because it provides a way to evaluate the effectiveness of each feature with respect to the overall effectiveness of a set of features. The inventive process allows users to choose the most effective features for expert system processing.
The present invention also provides users with the ability to create, test, and refine expert system decision trees utilizing their library of features. Such decision trees contain different branches based on different values of selected key features. Ultimately, all branches on the decision tree end with an occupant-type classification determination. The occupant-type classification, along with the vector of features, is sent to the confidence factor extractor. The confidence factor extractor uses both the occupant-type classification and the vector of features to generate a confidence factor number representing the probability that the classification determination is accurate.
The ability to generate a confidence factor indicating the probability that a particular classification determination is accurate is also a substantial advantage of the present invention. A confidence factor is mathematically derived from testing data obtained from the particular expert system classifier with its particular algorithm and set of features. A confidence factor is generated both through experimental test data and analytical means.
The invention is intended to maximize the potential benefits of what is referred to in the statistical arts as data mining. A single expert system classifier can implement a decision tree utilizing numerous different features. The invention can contain numerous expert system classifiers, with each expert system classifier utilizing multiple features. In all multiple-expert system embodiments of the invention, a weighted class predictor and confidence factor extractor are used to generate one overall classification determination, and one confidence factor relating to that overall determination. In a multiple expert system embodiment, each expert system could use identical feature sets, mutually exclusive features sets, or feature sets with some overlap in relation to other expert system classifiers.